How Challenging is Sarcasm versus Irony Classification?: An Analysis From Human and Computational Perspectives

Sarcasm and irony, although similar, differ in that sarcasm has an impact on sentiment (because it is used to ridicule a target) while irony does not. Past work treats the two interchangeably. In this paper, we wish to validate if sarcasm versus irony classification is indeed a challenging task. To this end, we use a dataset of quotes from English literature, and conduct experiments from two perspectives: the human perspective and the computational perspective. For the former, we show that three human annotators have lower agreement for sarcasm versus irony as compared to sarcasm versus philosophy. Similarly, sarcasm versus irony classification performs with a lower F-score as compared to another classification task where labels are not related: sarcasm versus philosophy classification. Another key point that our paper makes is that features designed for sarcasm versus non-sarcasm classification do not work well for sarcasm versus irony classification.